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Abstract 

Symmetrical Multilevel Diversity Coding (SMDC) is a network compression problem 
introduced by Roche (1992) and Yeung (1995). In this setting, a simple separate coding 
strategy known as superposition coding was shown to be optimal in terms of achieving 
the minimum sum rate (Roche, Yeung, and Hau, 1997) and the entire admissible rate 
region (Yeung and Zhang, 1999) of the problem. The proofs utilized carefully constructed 
induction arguments, for which the classical subset entropy inequality of Han played 
a key role. This paper considers a generalization of SMDC for which, in addition to 
the randomly accessible encoders, there is also an all-access encoder. It is shown that 
superposition coding remains optimal in terms of achieving the entire admissible rate 
region of the problem. Key to our proof is to identify the supporting hyperplanes that 
define the boundary of the admissible rate region and then builds on the result of Yeung 
and Zhang on a generalization of Han's subset inequality. As a special case, the {R(),Rs) 
admissible rate region, which captures all possible tradeoffs between the encoding rate 
Rq of the all-access encoder and the sum encoding rate Rg of the randomly accessible 
encoders, is explicitly characterized. To provide an explicit proof of the optimality of 
superposition coding, a new sliding-window subset entropy inequality is introduced and 
is shown to directly imply the classical subset entropy inequality of Han. 



1 Introduction 



Symmetrical Multilevel Diversity Coding (SMDC) is a network compression problem intro- 
duced by Roche [1] and Yeung [2]. In this setting, there are a total of L independent discrete 
memoryless sources {Si, . . . , Sl), where the importance of the source 5"/ is assumed to decrease 
with the subscript /. The sources are to be encoded by a total of L randomly accessible en- 
coders. The goal of encoding is to ensure that the number of sources that can be nearly 
perfectly reconstructed grows with the number of accessible encoders. More specifically, denote 
by f/ C := {1, . . . , L} the set of accessible encoders. The realization of U is unknown a 
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priori at the encoders. However, the sources {Si,...,Sa) need to be nearly perfectly recon- 
structed whenever \U\ > a. The word "symmetrical" here refers to the fact that the sources 
that need to be nearly perfectly reconstructed depend on the set of accessible encoders only via 
its cardinality. The rate allocations at different encoders, however, can be different and are not 
necessarily symmetrical. 

A natural strategy for SMDC is to encode the sources separately at each of the encoders (no 
coding across different sources) known as superposition coding [2]. To show that the natural 
superposition coding strategy is also optimal, however, turned out to be rather nontrivial. The 
optimality of superposition coding in terms of achieving the minimum sum rate was established 
by Roche, Yeung, and Hau [3]. The proof used a carefully constructed induction argument, for 
which the classical subset entropy inequality of Han [TJ Ch. 17.6] played a key role. Later, the 
optimality of superposition coding in terms of achieving the entire admission rate region was 
established by Yeung and Zhang [1]. Their proof was based on a generalization of Han's subset 
inequality, which was established by carefully combining Han's subset inequality with several 
highly technical results on the analysis of a sequence of linear programs. 

In this paper, we consider a generalization of SMDC for which, in addition to the randomly 
accessible encoders, there is also an all-access encoder. More specifically, in this new setting, 
a total of L + 1 independent discrete memoryless sources {Sq, Si, ... , Sl) are to be encoded by 
a total of L + 1 encoders. While encoders 1 to L are randomly accessible encoders as before, 
encoder is assumed to be an all-access encoder. Mathematically, if we denote hj U ^ ^II 
the set of randomly accessible encoders whose outputs are actually available at the decoder, 
then the set of accessible encoders at the decoder is {0} U U. As before, the realization of U 
is unknown a priori at the encoders. However, the sources (S'o, Si, ... , Sa) need to be nearly 
perfectly reconstructed whenever \U\ > a. 

Note that in the above setting, the source S'o needs to be nearly perfectly reconstructed 
whenever encoder is accessible. By our assumption, encoder is an all-access encoder. Hence, 
to minimize the encoding rates, there is no need to encode the source S'o using any of the 
randomly accessible encoders. If the encoding rate at encoder is set to be the entropy rate 
of the source S'o, then the sources {Si, . . . , Sl) must be encoded by the randomly accessible 
encoders 1 to L. In this case, the problem reduces to the original setting of Roche [T] and 
Yeung [2], for which superposition coding is known to be optimal [2111] • The main issue that 
we are concerned with is whether superposition coding will remain optimal when the encoding 
rate of the all-access encoder is greater than the entropy rate of the source S'o. 

The main result of the paper is that superposition coding remains optimal in terms of achiev- 
ing the entire admissible rate region of SMDC even with the addition of an all-access encoder. 
Key to our proof is to identify the supporting hyperplanes that define the boundary of the 
admissible rate region and then builds on the result of Yeung and Zhang on the generalization 
of Han's subset inequality. As a special case, the {Rq, Rs) admissible rate region, which cap- 
tures all possible tradeoffs between the encoding rate -Ro of the all-access encoder and the sum 
encoding rate Rs of the randomly accessible encoders, is explicitly characterized. To provide an 
explicit proof of the optimality of superposition coding, a new sliding-window subset entropy 
inequality is introduced and is shown to directly imply the classical subset entropy inequality 
of Han. 

The rest of the paper is organized as follows. The formal statement of the problem and the 
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Figure 1: SMDC with an all-access encoder and L randomly accessible encoders 1 to L. A 
total of L + 1 independent discrete memoryless sources (5*0, Si, ... , Sl) are to be encoded at the 
encoders. The decoder, which has access to encoder and a subset U of the randomly accessible 
encoders, needs to nearly perfectly reconstruct the sources (6*0, 5*1, ... , S\u\) no matter what the 
realization of [/ is. 

main result of the paper are summarized in Section O A proof of the main result is provided 
in Section [3l Discussions on the {Ro,Rs) admissible rate region and the sliding- window subset 
entropy inequality are provided in Section HI Finally, in Section |5] we conclude the paper with 
some remarks. 

2 Problem Statement and Main Result 
2.1 Problem Statement 

As illustrated in Figure [H the problem of SMDC with an all-access encoder consists of: 

• a total of L -|- 1 independent discrete memoryless sources {S'aft]}^;^, where a = 0, 1, . . . , L 
and t is the time index; 

• a set of L -|- 1 encoders (encoders to L); 

• a decoder who has access to a subset {0} UU of the encoder outputs, where U C fi^. 

The realization of U is unknown a priori at the encoders. However, no matter which U actu- 
ally materializes, the decoder needs to nearly perfectly reconstruct the sources {Sq, Si, ... , So) 
whenever \U\ > a. 

Formally, an [n, [Mq, Mi, . . . , M^)) code is defined by a collection of L+1 encoding functions 

L 

er.l[S2-^{l,...,Mi}, V/ = 0,1,...,L (1) 
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and 2 decoding functions 



du:{l,...,Mo}xl[{l,...,Mi} 



(2) 



leu 



a=0 



A nonnegative rate tuple {Rq, Ri, . . . , Rl) is said to be admissible if for every e > 0, there exits, 
for sufficiently large block length n, an (n, (Mq, Mi, . . . , M^)) code such that: 

• (Rate constraints at the encoders) 



-hgMi<Ri + e, V/ = 0,1,...,L; 

n 



(3) 



(Asymptotically perfect reconstructions at the decoder) 

Pr {duiXo, Xu) ^ (So, Si, ... , S|f;|)} < e, Vf/ C (4) 
where Sq, := Xi = ei{So, Si, . . . , S^) is the output of encoder /, and Xu := 

{Xr.le U}. 



The admissible rate region IZ is the collection of all admissible rate tuples {Rq, Ri, . . . , Rl)- 



2.2 Superposition Coding Rate Region 

As mentioned previously, a natural strategy for SMDC is superposition coding, i.e., to encode 
the sources separately at the encoders and there is no coding across different sources. Formally, 
the problem of encoding a single source Sa can be viewed as a special case of the general 
SMDC problem where the sources Sm are constants for all m 7^ a. In this case, the source Sa 
needs to be nearly perfectly reconstructed whenever the decoder can access at least a randomly 
accessible encoders in additional to the all-access encoder. Thus, the problem is essentially to 
transmit Sa over an erasure channel, and the following simple source-channel separation scheme 
is known to be optimal (whether the all-access encoder exists or not) [Il[2]: 

• First compress the source sequence Sa into a source message W using a lossless source 
code. It is well known [71 Ch. 5] that the rate of the source message W can be made 
arbitrarily close to the entropy rate H{Sq.) for sufficiently large block length n. 

• Next, the source message W is encoded at encoders to L using a maximum distance 
separable code [S]. It is well known [T1I2] that the source message W can be perfectly 
recovered at the decoder whenever 

i?o + 5^i?^>i^(5a), Vt/G^i") (5) 
leu 

where Q,^^^ is the collection of all subsets of il^ of size a. 
We summarize the above result into the following proposition. 
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Proposition 1. The admissible rate region for encoding a single source Sa is given by the 
collection of all nonnegative rate tuples {Rq, Ri, . . . , Rl) satisfying ([5]). 

By Proposition [H tlie superposition coding rate region TZgup for SMDC with an all-access 
encoder is given by the collection of all nonnegative rate tuples {Rq, Ri, . . . , Rl) such that 

for some nonnegative r\°'\ a = 0, 1, . . . , L and / = 0, 1, . . . , L, satisfying 

r(") + ^r(")>i7(5„), ^UeUf. (7) 

i&u 

Note that in theory, an explicit characterization of the superposition coding rate region TZsup 
can be obtained by eliminating r'f \ a = 0, 1, . . . , L and / = 0, 1, . . . , L, via a Fourier-Motzkin 
elimination from (|6]) and ([7]). The elimination process, however, becomes unmanageable even 
for moderate L, as there are simply too many equations involved. 



2.3 Main Result 

The main result of the paper is that superposition coding can achieve the entire admissible rate 
region TZ of SMDC with an all-access encoder, as summarized in the following theorem. 

Theorem 1. 

TZ = Tlsup- (8) 

A detailed proof of the theorem is provided in Section |3l Below we summarize the main 
technical ingredients of the proof. First, our proof relies on the following characterization of 
the superposition coding rate region Rsup- Let (Ai, . . . , Al) be a nonnegative vector in and 
let fa be the optimal value of the linear program 

subject to Z]r7<=t7(") r/giCQ(f/) < A;, V/ = 1,...,L (9) 



Ca{U) > 0, Vf/ G 1] 



(a) 
L 



for a = 1, . . . , L. Denote by IZ* the collection of all nonnegative rate tuples {Rq, Ri, . . . , Rl) 
satisfying 

Ro > H{So) (10) 

and 

L m L 

/„/?o + 5ZAzi2z>/^5^i7(^J+ J2 fo^H{Sa) (11) 

1=1 0=0 o=m+l 

for all m = 1, . . . , L and all nonnegative (Ai, . . . , Al) in M.^ . 
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Proposition 2. 



TZ =TZ* 



(12) 



A proof of the proposition is provided in Section 13. 1[ The proof uses the fact that TZsup is 
a polyhedron with polyhedral cone being the nonnegative orthant in M-^"*"^ and hence can be 
completely characterized by the supporting hyperplanes 

L 

J]A;i?^>/, V(Ao,Ai,...,Ai) >0 (13) 

1=0 

where 

L 

/= min y^XiRi. (14) 

{Ro,Ri,...,RL)&n,up 

We then complete the proof by showing that for any nonzero (Ai, . . . , Al), the faces of TZgup are 
only determined by the supporting hyperplanes with Aq = fm for m = 1, . . . , L. 

The second key ingredient of our proof is the following generalization of the classical subset 
entropy inequality of Han, which was first established in [H Theorem 3] . 

Proposition 3 (Generalized Han's subset inequality). For any nonnegative (Ai, . . . , Xl) in 'R^ , 
there exists a set of Ca '■= {caiU) : U G ^^l^}, a = 1,. . . ,L, for which each Ca is an optimal 
solution to the linear program (|9]) and such that 

^ ci{U)H{Xu)> C2{U)H{Xu)>---> Yl cl{U)H{Xu) (15) 

for any collection of L jointly distributed random variables (Xi, . . . ,Xl). 

In Proposition [3] was established via a delicate combination of Han's subset inequality 
and several highly technical results on the analysis of the linear program ([9]). In Section 13.21 
we provide a more structured proof which completely separates the entropy argument from the 
analysis of the linear program ([9]). The proof is based on a subset entropy inequality recently 
established by Madiman and Tetali [6] and an analysis result on the linear program ([9]) lifted 
from the original proof of Yeung and Zhang 

The following corollaries will be directly used in the proof of Theorem [H 

Corollary 1. For any nonnegative (Ai, . . . , A^) in MJ" , 

/i > 2/2 > ■ • • > LfL. (16) 

In particular, 

/i > /2 > • ■ ■ > /l. (17) 

Proof. Let Cq,, a = 1, . . . , L, be a set of optimal solutions to the linear program ([9]) and such 
that the inequality chain f|T5|) holds for any collection of L jointly distributed random variables 
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{Xi, . . . ,Xl). In particular, let (Xi, . . . ,Xl) be a collection of L independent and identically 
distributed random variables. We have H{Xu) = aH{Xi) for all U G fi^"'' and 

J2 c^{U)H{Xu) = aH{X,) c^{U) = aUH{X,) (18) 

for all a = 1,...,L. Here, the second equality is due to the fact that Cq, is optimal so 
J2u^q(°'1 '^a{U) = fa- Substituting f|T8l) into the inequality chain f lT5|l and dividing each term 
by H{Xi) give the inequality chain (|T6l) . 

For any a = 2, . . . , L, note from the inequality chain (1T6|) that 

fa^l > > U (19) 

a — I 

This proves the inequality chain f fT7|) . □ 

Corollary 2. For any nonnegative (Ai, . . . , Al) m M^, i/iere exists a set of Ca, a = 1, ■ ■ ■ ,L, 

for which each Ca is an optimal solution to the linear program (Q and such that 

ciiU)H{Xo,Xu\T)> J2 C2{U)H{Xo,Xu\T)>---> ^ cl{U)H{Xo, Xu\T) (20) 

for any collection of L + 2 jointly distributed random variables (Xo,Xi, . . . ,Xl,T). 

Proof. Given T = t, apply Proposition [3] with the vector-valued jointly distributed random 
variables ((Xq, Xi), . . . , (Xq, X^)). We have 



E^ef^a) c,{U)H{Xo,Xu\T = t) > 



L 



(21) 



Ec/enf C2{U)H{Xo,Xu\T = t) > ■ ■ ■ > E^^^w cl{U)H{Xo, Xu\T = t). 
Averaging over t completes the proof of the corollary. □ 



3 Proof of the Main Result 
3.1 Proof of Proposition [2] 

First note that the superposition coding rate region TZgup as described by and ([7j) is a 
polyhedron with polyhedral cone being the nonnegative orthant in so we can write TZsup 

as the collection of the nonnegative rate tuples {Ro, -Ri, . . . , Rl) such that 



X^Azi?z>/ (22) 



1=0 



for all (Ao, Ai, . . . , A^) > 0, where 



L 

f= min XiRi. (23) 
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Substituting (|6]) into (123|) . we can write the objective function of the optimization problem as 

t A,ft = t 1 '■;"') = t (t ^"•!"') . (24) 

1=0 1=0 \ a=0 / a=0 \l=0 1 

Note from ([7]) that the constraints that define the superposition coding rate region 'R-sup are 
completely separated for different a. Hence, the optimization problem ( 123|) can be solved by 
solving for each a = 0, . . . , L the optimization problem 

min Yi=o ^i^f^ 
subject to T^f \Y.i^^Tf''>E{S^), Vf/Cfi(.") (25) 
r{") > 0, V/ = 0, 1, . . . , L. 

Let be the optimal value of the optimization problem (!25|) . Clearly, /q = \qH{S^. For 
a = 1, . . . ,L, since the optimization problem f l25p is linear, by strong duality is also the 
optimal value of the dual program 



max 



subject to J2u&n{ 

a) Cq, (U) < Xo 



(26) 

^c/Gni"\c/3iC"(f/) < A«, V/ = 1,...,L 
Ca{U) > 0, Vf/ G n^L^. 

Note that if the constraint Y1u^q('^> Ca{U) < Aq is inactive, it can be removed from the dual 
program (126|) . In this case the optimal value = faH{Sa), where is the optimal value of 
the linear program (|9]). On the other hand, if the constraint CaiU) < Aq is active, the 

optimal value = XqH^So)- Combing these two cases, we have 

= min(Aoif(5„), f^H{S^)) = min(Ao, fa)H{S^) (27) 
for tt = 1 , . . . , L and hence 

L L 

/ = = ^O^(^O) + E^i^(^0,/a)i?(^a). (28) 

a=0 a=l 

Substituting ( 128|) into (122|) . we conclude that the superposition coding rate region TZsup is given 
by the collection of the nonnegative rate tuples (i?o, -Ri, • • • , Rl) such that 

L L 

J2 \Ri > ^oH{So) + J2 ^i^(^o, /a)i^ (^a) (29) 

1=0 a=l 

for all (Ao,Ai,...,Al) > 0. 

To show that TZgup ^ T^*, let (i?o, -Ri, • • • , -Rl) be a nonnegative rate tuple in TZgup that 
satisfies (!29|) for all (Aq, Ai, . . . , Al) > 0. Note that when (Ai,...,Al) = 0, = for all 
a = 1, . . . ,L. Thus, let (Aq, Ai, . . . , Al) = (1, 0, . . . , 0) and we have from ( 129|1 that the rate 
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tuple (i?o, Ri, • • • , Rl) must satisfy the inequality (fTOjl . Furthermore, for any m = 1, . . . , L, by 
the inequality chain ( IT7|) we have 

L m L 

^min(/^,/„)i7(5„) = /„,J]i7(^„)+ ^ (30) 

a=l a=l a=m+l 

Thus, let Ao = fm and we have from fl29l) and fl30l) that the rate tuple {Ro,Ri, . . . ,Rl) must 
also satisfy the inequality (fTT]) for all m = 1, . . . , L. This proves that (i?o, -Ri, • • • , -Rl) € 7^* 
and hence T^-^^ip C TZ*. 

To show the reverse relationship 71* C T^^^p, let {Rq, Ri, . . . , R^) be a nonnegative rate 
tuple in TZ* that satisfies flTU]) and flTTl) for all m = 1, . . . , L and all nonnegative (Ai, . . . , A^) in 
R-^. To show that the rate tuple {Rq, Ri, . . . , Rl) must also satisfy the inequality ( 129|) for all 
(Ao, Ai, . . . , Al) > 0, let us consider the following three cases separately. 

Case 1: Aq > /i- In this case, note that the inequality ( ITTjl with m = 1 can be written as 

L L 

hRo + J2 ^iRi > fiH{So) + fc^H{S^). (31) 

1=1 a=l 

From the inequalities (|3T|) and f lTOj) . we have 



•^0-Ro + ^ -^«-R« = (^0 — fi)Ro + I /i-Ro + ^ -^z-R/ 1 
1=1 \ 1=1 / 



(32) 



> (Ao - h)H{S,) + \^\H{S,) + ^ /«i/(5J j (33) 

L 

= XoH{So) + Y,UH{S^) (34) 

which is exactly the inequality ( l29ll with Aq > /i- 

Case 2: Aq < fi- In this case, note that the inequality ( iTTl) with m = L can be written as 

1=1 a=0 

From the inequality (!35|) . we have 



Aoi?o + $^A,i?, = ^^/^i?o + ^A,/2,j + (^1-^^ J]Azi?; 



(36) 



> ^ 



hRo + Y^i^^ (37) 



(3J 



AoJ]i/(5«) (39) 



a=0 



which is exactly the inequahty (!29|) with Aq < fi- 

Case 3: Aq G [/r+i, fr) for some integer r between 1 and L — 1. In this case, note that the 
inequahty ( ITTl) with m = r and m = r + 1 can be written as 



frRo + Y^kRl > frJ2H(^-)+ fo'HiSa) (40) 

1=1 a=0 a=r+l 

L r+1 L 

and fr+iRo + J2^i^i ^ fr+iY,H{Sa)+ Yl f-H{S^) (41) 



1=1 0=0 a=r+2 

respectively. From the inequalities (140!) and (14T|) . we have 



V Q=0 o=r+l / 

^ r+1 L \ 

/.+iEif(^«)+ E /.f^(5«) (43) 



Ao 


- /r+1 


fr- 


- /r+1 


Ao 


- /r+1 




~ /r+1 


/r 


-Ao 


fr- 


~ /r+1 



a=0 a=r+2 
L 



XoJ2HiSa)+ Y f-H{S^) (44) 



a=0 a=r+l 



which is exactly the inequality (!29|) with Aq G [/r+i, fr)- 

Combining these three cases proves that the rate tuple {Rq, Ri, . . . , Rl) G TZsup and hence 
71* C TZsup- Combined with the previous result TZsup C 7?.*, we conclude that TZsup = T^*- This 
completes the proof of Proposition [2l 

3.2 Proof of Proposition [3] 

Note that it is sufficient to prove that for any nonnegative (Ai, . . . , A^) in IR-^, any a = 2, . . . , L, 
and any Cq, which is an optimal solution to the linear program (Q, there exists a Ca-i which is 
also optimal for the linear program 

max Eygn<°-'' ^°-i(^) 
subject to Eygf7("-i),y3iCc,_i(V^) < A;, V/ = 1,...,L (45) 



and such that 

J2 c^-i{V)H{Xv)> Ca{U)H{Xu) (46) 



for any collection of L jointly distributed random variables {Xi, . . . , X^). To prove the existence 
of such let us recall the following two results from the literature. 
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First, a subset entropy inequality from [6l Theorem I'] which we rephrase here as follows. 
Consider a hypergraph {U, V) where U C 1]^;^ is a finite ground set and V is a collection of 
subsets of U. A function g : V ^ M"^ is called a fractional cover of {U, V) if it satisfies 

9{V)>1, Vzet/. (47) 

Lemma 3 (A subset entropy inequality of Madiman and Tetali). Let {U,V) be a hypergraph 
where the ground set U C VLl, and let g be a fractional cover of {U, V). Then 

H{Xu)<Y,9iy)H{Xv) (48) 

for any collection of L jointly distributed random variables (Xi, . . . iXl). 

The second one is an analysis result on the linear program ([9]), which we lift from the proof 
of [H Theorem 3]. Consider the hypergraph ([/, V(/) where U G and Vu '■= {V G : 
V (^U}. Let (7f/ be a fractional cover of ([/, Vu) and let g^f> := {gu ■ U e ^S"^}. 

Lemma 4. For any nonnegative (Ai, . . . , Xl) in M.^ , any a = 2, . . . , L, and any Ca which is an 
optimal solution to the linear program (Q with the optimal value /a > 0, there exists a set of 
fractional covers such that Ca-i = {ca-i{V) : V G 1]^"^^^} where 



c^^,{V)= ^c.{U)gu{V) (49) 

U£n'-^\uDV 

is an optimal solution to the linear program ( l45l) . 

A proof of Lemma m is provided in Appendix [Al When = 0, Ca{U) = for all U G ^^j^^ ■ 
Thus, any Ca-i which is optimal to the linear program ( l45l) trivially satisfies 

Y Ca-i{V)H{Xv)> J2 Ca{U)H{Xu) = 0. (50) 
When /q, > 0, combining (jl8|) and fHOj) gives 



J2 UU)H{Xu) < Yl ^°^^) 



Y gu{v)H{x, 



(o-l) 



E 



-,("-1) 



(51) 



H{Xv) (52) 



Y (^c.{U)gu{V) 
\uen'i\uDV j 

Y c^-iiV)HiXy) (53) 



for any collection of L jointly distributed random variables (Xi, . . . ,Xl). This completes the 
proof of Proposition O 
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3.3 Proof of Theorem [T] 

Since we naturally have TZsup ^ Tl-, to show that TZsup = 'R-, it is sufficient to show that TZ C 
TZsup- To show that 71 C TZsup, we need to show that any admissible rate tuple {Rq, Ri, . . . , Rl) 
must satisfy the inequalities ( ITOl) and (fTTl) for all m = 1, . . . , L and all nonnegative (Ai, . . . , Xl) 
in M^. The fact that any admissible rate tuple {Rq, Ri, . . . , Rl) must satisfy the inequality ffTOj) 
follows directly from Proposition [1] for encoding the single source Sq. 

To show that any admissible rate tuple {Rq, Ri, . . . , R^) must satisfy the inequality ffTTj) 
for all m = 1, . . . ,L and all nonnegative (Ai, . . . , Al) in M^, let c^, a = 1, . . . , L, be a set of 
optimal solutions for the linear program ([9]) and such that the inequality chain ( l20l) holds for 
any collection of L + 2 jointly distributed random variables {Xq, Xi, . . . , Xl, T). Note that for 
a = 1, the optimal solution for the linear program (Q is unique and is given by 

ci{{l}) = Xi, V/ = 1,...,L. (54) 

We thus have 

L 

UnRo + J2ci{{l})nRi (55) 
1=1 

L 

U{H{Xo) - ne) + ^ Ci({/})(iJ(XO - ne) (56) 
1=1 

L 

UH{Xo) + ^ ci{{l})H{Xi) - n(/i + U)e (57) 
1=1 

UH{Xo)+ cUU)H{Xu)-n{h + U)e (58) 

J2 cUU){H{Xo) + H{Xu))-n{h + U)e (59) 
^ c^(f/)/J(Xo,X^)-n(A + /Je (60) 

where f l56|) follows from the rate constraint ([3]), fl57|) follows from the fact that Ci is an optimal 
solution so fi = J^iLi ^ii{^}) y dSH]) follows from the fact that 

L 

Y^c,{{l})H{Xi)> cUU)H{Xu) (61) 

as specified by the inequality chain f l2U]) . f lS^ follows from the fact that is an optimal solution 
so fm = ^^^Q(rn) CmiU), and (160|) follows from the independence bound on entropy. 

For any U G by the asymptotically perfect reconstruction constraint ^ and Fano's 

inequality we have 

m 

if(S^|Xo,X^) < l+ne5^1og|5,|. (62) 

0=0 



n 



fmRo + Az-R/ j 



> 



> 



> 
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By the chain rule for entropy, we have 

H{Xo,Xu) = if(Xo,X[/, S™) - if(S[J^|Xo,X{/) 

= H{S^) + H{Xo,Xu\S^)-H{S^\Xo,Xu) 

m 

Y^HiS^) + H{Xo,Xu\S^) - H{S^\Xo,Xi 



n 



a=0 
m 



> nJ2H{S^) + H{Xo,Xu\S^)- l + ne^log|5, 



(63) 
(64) 

(65) 
(66) 



a=0 



a=0 



where (165|) is due to the fact that So, Si, . . . , Sl are independent memoryless sources, and (!66l) 
follows from (!62l) . Substituting (!66|) into (!60!) . we have 



n 



URo + J2^i^i] ^ 5^ c„(f/) n5];//(^,) + i/(Xo,Xf7|S^ 



(m) 



«=0 



1 +ne^log|>SQ 



nifi + /m)e 



(67) 



a=0 



/„^/7(5J+ ^ c„(f/)//(Xo,X^|S: 

/ m 

fm + neifmY log |5a| + fl + f, 



{61 



a=0 



To proceed, let us show, via an induction, that for any admissible code with encoder outputs 
{Xi}f^Q, any nonnegative (Ai, . . . , A^,) in R^, and any m = 1, . . . , L, we have 

L L 

Cm{U)H{Xo,Xu\S^)>n ^ fc^H{S^) - ^ /a (1 + nelog |5J) . (69) 



(m) 



a=m+l 



Q=m+1 



First, when m = L, the inequality 0691) is trivial as the right-hand side of the inequality is zero. 
Next, assume that the inequality fl^^ holds for some m = I, i.e. 



Y Q{U)H{Xo,XuK)>n Y fc^H{S^)- Y fa{l+ne\og\S^\ 



(70) 



(0 



a=l+l 



a=l+l 



For any U G we have 



= H{Xq,Xu 






(71) 


= H{Xq,Xu 




- H{Si\S',~') + H{Si\Xo,Xu,S'o~') 


(72) 


< H{Xq,Xu 




-HiSi\S',~') + HiSi\Xo,Xu) 


(73) 


< H{Xq,Xjj 




-HiSi\S',-') + il + ne\og\Si\) 


(74) 


= H{Xo,Xu 




-nH{Si) + {l + nehg\Si\) 


(75) 
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where (173!) follows from the fact that conditioning reduces entropy, (17^ follows the asymptoti- 
cally perfect reconstruction constraint (|4]) and Fano's inequality so we have 

H{Si\Xo,Xu) <l + nehg\Si\ (76) 

and (175|) follows from the fact that So, Si, . . . , Sl are independent memoryless sources. Multi- 
plying both sides of the inequality (1751) by ci{U) and summing over all U G Qf \ we have 

J2 ci{U)H{Xo,XuK~') 



(0 



> 5^ Q(t/)(i/(Xo,Xc;|S|,)+n/7(50-(l+nelog|5,|)) (77) 
= ^ Q(f/)if(Xo,Xc;|S|,) + n/,i7(50-/Kl + ^elog|5n) (78) 

L L 

> n ^ UH{S^)- ^ /„(l + nelog|5,|) + n/;i7(^0-/z(l + ^elog|'5d) (79) 



^ /,i/(5J - ^ (1 + ne log \S^\) (80) 



«=Z+1 a=i+l 
L L 

n 

where fl79|) follows from the induction assumption fl70|) . Finally, by the inequahty chain fl20l) 
we have 

^ Q_i(t/)if(Xo,Xc;|S[,-i) > ^ ci{U)H{Xo,Xu\S',~') (81) 

L L 

> n^/„i/(^J-5^/,(l + nelog|5„|). (82) 

a=i a=l 

This proves that the inequality ( 1691) also holds for m = I — 1. 

Substituting fl69l) into fl68l) and dividing both sides of the inequality by n, we have 

L m L 

/™i?o + $^Azi?z>/^^i7(5,)+ faH{S^)-6Un,e) (83) 

1=1 a=0 a=m+l 

where 

L / m L \ 

5^(n,e) =n"^^/« + e I /™^log|5«|+ ^ log + /i + | . (84) 

a=m \ 0=0 a=m+l / 

Let n — )■ oo and e — 0. Note that d^in, e) — for any nonnegative (Ai, . . . , A^) in and any 
m = 1, . . . , L. We thus have from fl83l) that 

L m L 

/™i?o + 5^A,i?,>/„^iJ(5J+ ^ faH{S^) (85) 

i=l Q=0 a=m+l 

for any admissible rate tuple {Rq, Ri, . . . , Rl). This proves that IZ C IZgup and hence completes 
the proof of Theorem [H 
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Ro 



L 




H(So) 








Figure 2: An illustration of the {Rq, Rs) admissible rate region TZ'. In general, the region is a 
(two-dimensional) polyhedron with polyhedral cone being the first quadrant and L + 2 faces. 

4 Discussions 

As shown in Section |3l our proof of the optimality of superposition coding for the entire ad- 
missible rate region relies on a characterization of the superposition coding rate region which 
involves solving a sequence of linear programs (j9]) for a = 1, . . . , L. For a general nonnegative 
(Ai, . . . , Xl) in M.^, the optimal solution of the linear program (|9]) cannot be written in closed- 
form for a = 2, . . . , L — 1. For the symmetrical situation where Ai = ■ ■ ■ = A^, however, it is 
straightforward to verify that 



for all a = 1, . . . , L. In this section, we discuss some ramifications of (15^ and ( 1S7|) . 

4.1 An Explicit Characterization of the (Rq^Rs) Admissible Rate 
Region 

First, let us use (IHTj) to establish an explicit characterization of all possible tradeoffs between 
the encoding rate Rq of the all-access encoder and the sum encoding rate Rg of the randomly 




(86) 



is an optimal solution to the linear program ([9]), giving the optimal value 




(87) 
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accessible encoders, as captured by the (i?o, Rs) admissible rate region of the problem. 
Formally, the (i?o, Rs) admissible rate region 71' is defined as 

7^' := I {Ro, Rs) • = (^0' ^1' • • • ' ^ ^1 (88) 

where TZ is the admissible rate region of the problem. Let Ti) be the collection of all nonnegative 
rate pairs (i?o, Rs) satisfying 



L 

Ro + t^s>Th{So.)+ V -H{Sa) (89) 

a=0 a=m+l 

for all m = 0, 1, . . . , L. We have the following explicit characterization of the {Ro, Rs) admissible 
rate region TZ' (see Figure [2] for an illustration). 

Theorem 2. 

7^' = nl (90) 

Proof. By Theorem [T] and Proposition [21 the admissible rate region TZ = TZsup = TZ*, so the the 
(i?o, Rs) admissible rate region 7Z' can be written as 

7^' := I {Ro, Rs) '-^' = ^1 (^0' Ru---,Rl) en*"^. (91) 

To show that 7^' C 7^^ let {Rq, . . . , Rl) be a nonnegative rate tuple in TZ*. By definition, 
{Rq, Rl, . . . , Rl) must satisfy (ITO!) and (ITT]) for all m = 1, . . . , L and all nonnegative (Ai, . . . , A^) 
in M^. Let Rg = Yld=iRi ^^'^ ^y (ITUI) . {Ro,Rs) must satisfy (jSHD for m = 0. Furthermore, let 
Ai = ■ ■ ■ = Al > and we have from fllip and fl87l) that (i?05 Rs) must also satisfy 

^i?o + XiRs > ^ f: //(5.) + E ^//(5.) (92) 

1=0 l=m+l 

for all m = 1, . . . , L. Dividing both sides of the inequality by LXi/m gives exactly the inequality 
(1891). We have thus proved that {Ro, Rs) G 7^"^ and hence 7^' C 7^^ 

To show the reverse relationship TZ'^ C 7Z', let us show that for any {Rq, Rg) G TZ\ the rate 
tuple 

{Ro, Rl, . . . , -Rl) = {Rq, Rs/L, . . . , Rg/L) (93) 
must be in TZ*. The fact that Ro > H{So) follows directly from the inequality flS^ with m = 0. 
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Furthermore, for any nonnegative (Ai, . . . , Al) in and any m = 1, . . . , L we have 

L L 



1=1 1=1 

L 



= /™i?o + ^5^ci({/}) (95) 
1=1 

= fmRo + jRs (96) 

> fmRo+'^Rs (97) 

(TTl \ 
Ro + jRs) (98) 

(m L \ 

a=0 a=m-\-l / 

m L f, 

= fmy^H{S^)+ V ^i/(5J (100) 

Q!=0 a=m+l 
m L 

a=0 a=m+l 

where (p7I) and f llOip are due to the inequahty chain (fTTl) so we have /i > m/^ > for all 
a = m + 1, . . . , L, and (199|1 follows from the fact that {Rq, Rg) G 71^ so 

i?o + T^.>E^(^")+ E -^(^")- (102) 

a=0 a—m+1 

We have thus proved that the rate tuple (l93l) must be in 7^* and hence TZ'^ C 7?.'. 

Combining the facts that TZ' C 7^1" and 7^^ C TZ' gives TV = TZ\ which completes the proof 
of the theorem. □ 



4.2 A Sliding- Window Subset Entropy Inequality 

By Theorem [21 the boundary of the {Rq, Rg) admissible rate region TZ' is determined by the 
supporting hyperplanes of the admissible rate region 7Z with Aq = 1 and Ai = ■ ■ ■ = Al = m/L 
for m = 0, 1, . . . , L. Therefore, to prove the optimality of superposition coding for the {Rq, Rs) 
admissible rate region TZ', one may only need to invoke the generalized Han's subset inequality 
for the special case where Ai = ■ ■ ■ = A^ > 0. Substituting (186|) into the inequality chain f|T5|) 
gives 

(103) 



Al 
L-1 




> 



(1) 



Al 
L - 1 
1 



(2) 
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Figure 3: An illustration of the sliding windows of length a when the integers 1, . . . , L are 
circularly placed (clockwise) based on their natural order. 



Dividing each term by AiL and using the fact that 

1 



r~ 


I 


a 1 




\ a — 






a / 



(104) 



L 

we have 



which is precisely the classical subset entropy inequality of Han. We thus conclude that Han's 
subset inequality is sufficient to prove the optimality of superposition coding for the {Rq, Rg) 
admissible rate region. 

Note, however, that when Ai = ■ • ■ = Al > 0, the optimal solution to the linear program 
([9]) is not unique for a = 2, . . . , L — 1. Below we provide a different set of optimal solutions to 
the linear program for a = 2,...,L — 1, based on which we construct a new subset entropy 
inequality. 

We shall start with the following notations. For any integer i, let us define 

, ., ( i mod L, if i mod L ^ ,^ 
■= L, if z mod L = 
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and for any i = 1, . . . , L and a = 1, . . . , L, let 

Pyf):=K(^ + l),..., (2 + «-!)}. (107) 

As illustrated in Figure El W^/"^ represents a sliding window of length a starting with i when the 
integers 1, . . . , L are circularly placed (clockwise or counter clockwise) based on their natural 
order. Let w{"^ = {W^/"^ : i G I^l} be the collection of the sliding windows of length a. It is 
straightforward to verify that when Ai = ■ ■ ■ = A^, the following solution is also optimal for the 
linear program (Q for a = 2, . . . , L — 1: 

^ ' \ 0, f/ G 9!^^\wr- 

Substituting fllOSp into the inequality chain f|T5|l suggests the following sliding-window subset 
entropy inequality, which we prove next. 

Theorem 3 (A sliding-window subset entropy inequality). For any collection of L jointly 
distributed random variables (Xi, . . . ^X^), we have 

L L L 

i=l i=l i=l 

The equalities hold when Xi, . . . are mutually independent of each other. 
Proof. Note that it is sufficient to show that for any a = 2, . . . , L, 

^E^(^H/<-)) >iilH{X^,.,). (110) 

i=l i=l 

Next, we shall prove flllOp via an induction argument and the submodularity of the entropy 
function. 

First, when a = 2, we have 

L L 



Y,H{X^w) = $^i^(X.) (Ill) 

1=1 

1 ^ 

= -Y,[HiX,) + H{X^,^,))] (112) 

i=l 
1 ^ 

> -^i7(X„X(,+i)) (113) 

1 ^ 



1=1 1=1 

L 



i=l 



where flllSp follows from the independence bound on entropy. Thus, the inequality flllOp holds 
for a = 2. 
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Next, assume that the inequahty f lllOp holds for ct = r — 1. We have 



L ^ L 

^if(X^(.-i)) = 7;Y1 



i=l 



i=l 
L 



> 



i=l 
L 



H{X^,,) + H{X 



(i + l> 



> 



i=l 



2); 

<i + l> 



i=l 



1 r -2 



i 

L 



2 r 



i=l 



where (11161) follows from the submodularity of entropy [H Ch. 14. A] 

H{Xu) + H{Xy) > H{Xuuv) + H{Xunv) 



(115) 
(116) 
(117) 
(118) 
(119) 

(120) 



for U = "^^ and V = Wj^''^'^' soUUV = W^^"' and f/ n V = Wj^''^"' , and ^19^ follows from 
the induction assumption 



Kr-2) 



1 ^ 1 ^ 

i=l ' i=l 



:i2ii 



Moving the second term on the right-hand side of (11191) to the left and multiplying both sides 
by 2/r gives 



1 1 



(122) 



i=l i=l 

We have thus proved that the inequality (lllOp also holds for a = r. 

Finally, note that when Xi, . . . , Xl are mutually independent, we have 



^ L L 

-Y,H{X^,^,) = Y,H{X,), Va = l,...,L. 

" i=i ' i=i 



This completes the proof of Theorem [3l 



(123) 



□ 



Just like the classical subset entropy inequality of Han, the sliding-window subset entropy 
inequality is also sufficient to prove the optimality of superposition coding for the (i?o, Rs) 
admissible rate region. In fact, Han's subset inequality can be derived from the sliding-window 
subset entropy inequality via a very simple permutation argument as follows. 

Let vr be a permutation on VL^. For any i = 1, . . . , L and a = 1, . . . , L, let 



W. 



(a) 



(124) 
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By Theorem [3], we have 



L L L 



i=l i=l j=l 



Averaging (I125p over all possible permutations n, we have 



(125) 



-y 



1=1 



TT i = l 



2), 

7V,i 



> ■■■> 



TT 7=1 



Note that for any a = 1 



, . . . , L, 



(126) 



5^^if(X^,.,) = L-a!(L-a)! i/(Xf;). 



TT i = l 



(127) 



Substituting 01271) into 01261) and dividing each term by L give the classical subset entropy 
inequality of Han (I1U5I) . 



5 Concluding Remarks 

This paper considered the problem of SMDC where, in addition to the randomly accessible 
encoders, there is also an all-access encoder. This is a natural extension of the original SMDC 
problem introduced by Roche [1] and Yeung [2] , for which superposition coding was shown to be 
optimal in terms of achieving the minimum sum rate [3] and the entire admissible rate region [1]. 
For this generalized setting, it was shown that superposition coding remains optimal in terms 
of achieving in terms of achieving the entire admissible rate region of the problem. Key to our 
proof is to identify the supporting hyperplanes that define the boundary of the admissible rate 
region and then builds on the result of Yeung and Zhang [1] on a generalization of Han's subset 
inequality. As a special case, the (i^o, Rs) admissible rate region, which captures all possible 
tradeoffs between the encoding rate Rq of the all-access encoder and the sum encoding rate Rs 
of the randomly accessible encoders, is explicitly characterized. To provide an explicit proof 
of the optimality of superposition coding, a new sliding-window subset entropy inequality is 
introduced and is shown to directly imply the classical subset entropy inequality of Han. We 
expect that the sliding-window subset entropy inequality will be useful for solvoing some other 
network compression and communication problems as well. 

As a side development, a more structured proof of the generahzed Han's subset inequality is 
provided, which completely separates the entropy argument from the analysis of the underlying 
linear programs. The proof is based on a subset entropy inequality recently established by 
Madiman and Tetali [6] and an analysis result on the underlying linear programs that we lift 
from the original proof of Yeung and Zhang [3]. We believe that this improved proof represents 
a better reflection on the technical nature of the generalized Han's subset inequality. 

Finally, we mention here that an "asymmetric" setting of the multilevel diversity coding 
problem was considered in the recent work [9], where the sources that need to be asymptotically 
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perfectly reconstructed depend on, not only the cardinality, but the actual subset of the encoder 
outputs available at the decoder. Unlike the symmetrical setting considered in [T]-0] and in this 
paper, as demonstrated in j9] for the case with three encoders, coding across different sources 
is generally needed to achieve the entire admissible rate region of the problem. 



A Proof of Lemma 4 



Without loss of generality, let us assume that 

Ai > A2 > ■■■ > Ai. (128) 
When L = 2, the optimal solutions to the linear program (jH]) are unique and are given by 

ci({/}) = A,, / = 1,2 (129) 

and 

C2({1,2}) = A2. (130) 
When /2 = A2 > 0, it is straightforward to verify that 

^7{i,2}({/}) = A,/A2, / = 1,2 (131) 

is a fractional cover of ({1, 2}, V{i,2}) and satisfies (jUj). 

Assume that a set of desired fractional covers Q^^^ exists for some L = N — 1 whenever 
fa > 0. To show that the desired fractional covers also exist for L = N when > 0, we shall 
consider the following three cases separately. In each of the cases below, we will construct a set 
of fractional covers = {gu : U G f^^^} which leads to the same Ca-i as those constructed 
in the proof of [H Theorem 3] . Therefore, the readers are referred to the proof of [H Theorem 3] 
for the optimality of such Ca-i- Instead, our focus is on proving that the so-constructed gu is 
indeed a fractional cover of {U,Vu) for each U E . 

Case 1: Ai < ■ In this case, let us consider for any U G fi^^ 

gu{V) = ^—, ^VeVu (132) 
a — 1 

so 

Ca-iiV)= V WVen^r'^. (133) 

Note that this gives the same Ca-i as [H Eq. (39)] so it is an optimal solution to the linear 
program f H5|) . Further note that for any i E U, \{V E Vu '■ V 3 i}\ = a — 1 so 

E 9{V) = 1. (134) 

We thus conclude that gu is a fractional cover of {U, Vu) for any U E fij^"*. This completes the 
proof of Case 1. 
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Case 2: Ai > ^^^3^. In this case, by [H Lemma 6] for any Cq, which is an optimal solution 
to the linear program (Q, Ca{U) > implies that U 3 1. Furthermore, by [H Lemma 8] 
Ca^i = {ca_i(f/) : U C n^^i := {2, . . . , A^}} where 

c„_i(f/) =c„({l}Ut/) (135) 

is an optimal solution to the linear program 

max Zlr7pA(°-i) ^"-i(^) 



N-l 



subject to Zlf/pnt"-!) j7=,/ < A;, V/ = 2, . . . , iV (136) 



Ca-i{U) > 0, me Q^'"^'' 



with the optimal solution f^-i = /« > 0. Thus, by the induction assumption there exists a set 
of fractional covers G^'-P — idu '■ ^ ^ ^!v-i^} such that Ca-2 = {ca-2{V) : V G fij^^^''} where 

5a-2iV)= Yl Ca-i{Ur9uiV) (137) 

is an optimal solution to the linear program 

max Zly6Q(^-2) Ca-2(y) 
subject to Ev^6q(— 2)y3, Ca-2(y) < Ai, V/ = 2, . . . , (138) 

c„-2(^)>o, v\/G(^;^:^ 

Consider for any U G f^^"* such that 1 G f/ 

I 'f V = {1} U 1/ for some 1/ 6 V, 

^ ^ [0, otherwise ^ ^ 

where f/ = ?7 \ {1}. For any U G fij^'' such that 1 ^ U, we can pick gu to be any fractional 
cover of {U, Vu) as Ca{U) = 0. Note that for any V G fij^ such that 1 G \^ 



c„_i(\/) = Yl (^c.{U)gu{V) (140) 

N-l ^'^^^ 

Y Ca^iiUyguiV) (142) 



= Ca-2{V) (143) 

where V = V \ {1}, and for any V G fij^"^'' such that 1 ^ V 

c^-i{V)= Yl Ca{U)gu{V)=0. (144) 

uen]^\uDV 
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This gives the same Ca-i as [H Eq. (46)] so it is an optimal solution to the linear program (H5|l . 
It remains to show that gu is a fractional cover of {U, Vu) for any U E il^^^ such that U 3 1. 
For any i E U \ {1}, we have 

E 9uiV)= Yl 9uiV)= Yl ~9uiV)>'^ (145) 
veVu,V3i veVu,v^{i,i} ve%,V3i 

and 

Y 9u{V)> Yl 9u{V)>l. (146) 

This completes the proof of Case 2. 

Case 3: ^2+---+>^n < ^, < ^2+---+^n _ jj^ ^-j^jg c^se, we shall need the following notations. For 

a—l ^ — a— 2 ' ° 

any U G Q^^^ and any integer r between 1 and a, denote by a(7(r) the smallest positive integer 
I such that 

|{l,...,/}nf/| =r. (147) 

Let 

iy.(?7) ■.= U\{au{T)} (148) 

so WT-(f/) G f^j^"^^ for all f/ G fij^"* and all integer r between 1 and a. Define a mapping 

^u,m,T '■ ^7v~^^ ~^ f*-*^ each t/ G ^^^\ each integer m between 2 and a, and each integer r 
between m and a by 

fe.„.,(V) := ( if ^ = WMf/) (149) 

0, otherwise 



where 



fc!"^ := A, -A, (150) 
and ~Xi := ^ c„([/), V/ = 1,...,L. (151) 



Let 



a-l 



m=2 

and consider the set of fractional covers = {gu : f/ G ^^^^} where 

9u{V) =(l-j-) + E E ^u,mAV)^ G Vc;. (153) 

This gives 



^.-i(v^) = (i-f) E ^+ E EEw(v^K(f/), vvGfi 



(a-l) 
TV 



(q) ^j^,^m=2T=m 

(154) 
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which is the same as [H Eq. (55)] and hence is an optimal solution to the hnear program (145|1 . 

)(") 

'AT 



It remains to show that gu is a fractional cover of {U, Vu) for any U G fif ^ 



Note that for any i E U 



veVu,V3i 
and 



E EEw(^) = EE E w(^) ase) 

= EE^^^^S — ^^WW^O (157) 

r, J a 

171=2 T=m 

= EH-^ ElK-W« (158) 

m=2 \r=m / 



m=2 



(160) 



where (11601) follows from [H Eq. (66)]. Combing fll55p and fll60p gives 

fa f< 



E 9u{V)>l-^ + ^ = l. (161) 



veVu,V3i 

We thus conclude that gu as defined in fll53p is indeed a fractional cover of {U,Vu) for any 
f/ G f^j^''. This completes the proof of Case 3. 
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